Skip to content

Add atomic journaled write to plugins; Add Siemens S7Comm - #75

Merged
thiagoralves merged 27 commits into
mainfrom
development
Jan 14, 2026
Merged

Add atomic journaled write to plugins; Add Siemens S7Comm#75
thiagoralves merged 27 commits into
mainfrom
development

Conversation

@thiagoralves

Copy link
Copy Markdown
Contributor

This pull request introduces several improvements and new features to the OpenPLC Runtime v4 project, focusing on enhanced plugin support, documentation, and build optimization. The most significant changes include the addition of a comprehensive developer guide, a new race-condition-free journal buffer API for plugins, and a complete build system for the native S7Comm plugin. There are also optimizations to Docker images and updates to the CMake build configuration.

Documentation and Developer Guidance

  • Added a new CLAUDE.md file with detailed instructions for building, testing, linting, architecture overview, code style, and compilation flow for OpenPLC Runtime v4. This provides clear guidance for developers and AI coding assistants.

Plugin System Enhancements

  • Introduced a journal buffer API for plugins, enabling race-condition-free writes to I/O buffers via new function pointers in plugin_types.h and their initialization in plugin_driver.c. All writes are applied atomically at the start of the next PLC scan cycle. [1] [2] [3] [4]
  • Updated plugin_driver.c to include and use journal_buffer.h, and provided wrapper functions for the new journal API.

Build System Improvements

  • Added a complete CMakeLists.txt for the native S7Comm plugin, including Snap7 and cJSON sources, correct include paths, compiler options, and platform-specific settings for building a self-contained shared library.
  • Updated the main CMake configuration to include journal_buffer.c in the PLC runtime build.

Docker and Development Environment Optimization

  • Added Docker-specific optimizations to both Dockerfile and Dockerfile.dev by cleaning up the apt cache after installation, reducing the final image size. [1] [2]

These changes collectively improve developer experience, plugin reliability, and build efficiency across the OpenPLC Runtime project.

thiagoralves and others added 27 commits January 12, 2026 17:47
- Add comprehensive implementation plan covering:
  - Analysis of OpenPLC v3 S7Comm/Snap7 implementation
  - Snap7 library API documentation
  - JSON configuration schema design
  - Plugin structure and data structures
  - 7-phase implementation roadmap
  - Testing strategy

- Add user guide with:
  - Configuration reference
  - Example configurations
  - S7 address mapping guide
  - Client connection examples
  - Troubleshooting guide

- Add default s7comm_config.json with standard mappings

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Phase 1 implementation of S7Comm plugin for OpenPLC Runtime v4:

- Include Snap7 library source code (v1.4.3) for self-contained build
- Add plugin source files (s7comm_plugin.cpp/h)
- Add CMakeLists.txt for standalone plugin compilation
- Implement plugin lifecycle: init, start_loop, stop_loop, cleanup
- Implement cycle hooks for buffer synchronization
- Register S7 system areas (PE, PA, MK) and data blocks (DB1, DB2, DB10, DB20, DB100, DB200)
- Handle big-endian conversion for S7 protocol compatibility
- Add event logging for client connections

Data block mapping (Phase 1 - hardcoded):
- DB1: bool_input (%IX)
- DB2: bool_output (%QX)
- DB10: int_input (%IW)
- DB20: int_output (%QW)
- DB100: int_memory (%MW)
- DB200: dint_memory (%MD)

Note: JSON configuration support will be added in Phase 2.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add build_native_plugins() function that automatically scans for native
plugins with CMakeLists.txt and builds them during installation. This
enables Docker images to include pre-built native plugins like s7comm.

The function:
- Scans core/src/drivers/plugins/native/ for CMakeLists.txt files
- Creates isolated build directories for each plugin
- Passes OPENPLC_ROOT to cmake for proper header resolution
- Copies built .so files to build/plugins/
- Provides summary of successful/failed builds

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add full JSON configuration support using embedded cJSON library:

Configuration features:
- Server settings (port, max_clients, timeouts, PDU size)
- PLC identity (name, module_type, serial_number, etc.)
- Dynamic data block allocation with configurable mappings
- System areas (PE, PA, MK) with configurable sizes
- Logging options (connections, data_access, errors)

Implementation changes:
- Add cJSON library (v1.7.18, MIT license) for JSON parsing
- Add s7comm_config.h with configuration structures
- Add s7comm_config.c with parser and validation
- Refactor s7comm_plugin.cpp to use configuration:
  - Dynamic memory allocation for data blocks
  - All server parameters from config
  - Support for all buffer types (bool, int, dint, lint)
  - 64-bit (LINT) support with swap64 endianness conversion
- Update CMakeLists.txt to include new source files

The plugin now reads s7comm_config.json at startup and dynamically
allocates and registers S7 data areas based on configuration.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Refactor buffer synchronization to use double-buffering pattern:
- Add shadow buffers for all S7 areas (system areas + data blocks)
- Add S7 mutex (pthread_mutex_t) to protect S7 buffers during sync
- Implement three-step sync in cycle_end:
  1. Lock mutex, copy S7 -> shadow (capture client writes)
  2. Sync shadow <-> OpenPLC (slow part, no S7 mutex held)
  3. Lock mutex, copy shadow -> S7 (publish new values)
- Remove sync from cycle_start (now no-op)

This allows S7 clients to read/write asynchronously from the main
PLC cycle with minimal mutex contention.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use Srv_GetStatus to check client count at the start of cycle_end.
If no clients are connected, return early without any mutex operations
or buffer copies, reducing overhead when S7 server is idle.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add JSON_SCHEMA.md with complete configuration reference for Editor
- Document all fields, types, constraints, and default values
- Add Editor UI recommendations and validation rules
- Update USER_GUIDE.md with double-buffering architecture section
- Add data flow diagram and related documentation links

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix Srv_GetStatus: use references (int&) not pointers (int*)
- Fix CMakeLists.txt: apply -Wno-class-memaccess only to C++ files
  using COMPILE_LANGUAGE generator expression

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add S7Comm native plugin entry to plugins.conf and plugins_default.conf
so the runtime will recognize and load the S7Comm server when a config
file is provided by the editor.

Plugin configuration format:
- name: s7comm
- path: ./core/src/drivers/plugins/native/s7comm/libs7comm_plugin.so
- enabled: 0 (disabled by default, enabled when config file is present)
- type: 1 (native plugin)
- config_path: ./core/src/drivers/plugins/native/s7comm/s7comm_config.json

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update path to match where the installer places the built library:
core/src/drivers/plugins/native/s7comm/build/plugins/libs7comm_plugin.so

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Srv_GetStatus client count check was incorrectly returning 0
even when clients were connected, causing buffer sync to be skipped.
Remove this optimization to ensure buffer sync always happens.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ication

Change the sync strategy to copy ENTIRE buffers in both directions:
- cycle_start: Copy ENTIRE S7 buffer -> OpenPLC buffer (all types)
- cycle_end: Copy ENTIRE OpenPLC buffer -> S7 buffer (all types)

This allows S7 clients to write to any location (inputs, outputs, memory).
Values actively driven by the PLC program or I/O drivers will naturally
overwrite S7 writes during the scan cycle, but S7 writes to other
locations (e.g., outputs used only as contacts) will persist.

Fixes issue where PLC outputs were being overwritten with zeros from
the S7 buffer.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This document describes a proposed journaling system for plugin writes
to OpenPLC image tables. Key features:

- All plugin writes go to a journal buffer instead of direct buffer access
- Journal is applied atomically at cycle_start with "last writer wins"
- Solves race conditions between asynchronous plugins (Python) and
  synchronous plugins (native with cycle hooks)
- Eliminates the "zero vs uninitialized" ambiguity
- No priority configuration needed - temporal ordering is deterministic

The document includes:
- Detailed architecture diagrams and data flows
- API specifications for both C and Python
- Thread safety model and mutex ordering
- Integration points with existing runtime
- Implementation phases with effort estimates
- Comparison with alternative approaches

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
feat: Add S7Comm server plugin for Siemens S7 protocol communication
…e 1)

Implement core journal buffer infrastructure:
- journal_buffer.h: Public API with write functions for all buffer types
- journal_buffer.c: Implementation with mutex-protected journal entries

Key features:
- Static buffer with 1024 max entries (~20KB memory)
- Sequence-numbered entries for last-writer-wins conflict resolution
- Thread-safe write functions for bool, byte, int, dint, lint types
- journal_apply_and_clear() for atomic application at cycle_start
- Emergency flush when buffer is full (respects lock ordering)
- Diagnostic functions for monitoring pending entries

This addresses race conditions between plugins writing to shared
image tables by routing all writes through a journal that is
applied atomically at the start of each PLC scan cycle.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Initialize journal buffer in plc_cycle_thread after image tables setup
- Call journal_apply_and_clear() at start of each PLC cycle
  (after acquiring buffer_mutex, before plugin cycle_start hooks)
- Cleanup journal buffer in unload_plc_program before clearing image tables

This ensures all plugin writes from the journal are atomically applied
at the start of each scan cycle, before plugins and the PLC program run.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ecture

docs: Add journal buffer architecture specification
Add journal buffer API for Python plugins:
- Add journal write wrapper functions in plugin_driver.c
- Set journal function pointers in plugin_runtime_args_t
- Add ctypes bindings in PluginRuntimeArgs Python class
- Modify GenericBufferAccessor to use journal writes
- Keep read operations unchanged (direct buffer access)

All Python plugin writes now go through the journal system
for race-condition-free buffer access.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove unnecessary mutex from simple setValues methods since
  journal writes are internally thread-safe
- Keep mutex for OpenPLCSegmentedHoldingRegistersDataBlock.setValues
  because it requires read-modify-write for partial DINT/LINT updates
- Add documentation explaining journal-based thread safety
- Update batch_processor.py with thread safety notes

Affected setValues methods:
- OpenPLCCoilsDataBlock: mutex removed (simple bool writes)
- OpenPLCHoldingRegistersDataBlock: mutex removed (simple int writes)
- OpenPLCSegmentedCoilsDataBlock: mutex removed (simple bool writes)
- OpenPLCSegmentedHoldingRegistersDataBlock: mutex kept (RMW needed)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement S7Comm plugin for Siemens S7 communication using Snap7 library.
Uses journal buffer system for race-condition-free writes to OpenPLC.

Key features:
- On-demand data sync via Snap7 RWArea callback (no cycle hooks needed)
- S7 client reads: acquire OpenPLC mutex, copy fresh data, release mutex
- S7 client writes: use journal_write functions (thread-safe, no mutex)
- Support for PE/PA/MK system areas and configurable data blocks
- JSON-based configuration for flexible I/O mapping
- Big-endian to little-endian conversion for S7 protocol

The S7 server thread runs independently without requiring cycle_start/
cycle_end hooks for data synchronization, similar to Python Modbus plugins.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Resolve merge conflicts in s7comm_plugin.cpp by keeping the journal-based
implementation. The S7Comm plugin now uses on-demand data synchronization
via the RWArea callback instead of cycle-based double-buffering.

Key changes from development:
- S7Comm plugin added to plugins.conf
- Install script updates

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The callback receives S7 protocol area codes (S7AreaPE=0x81, S7AreaPA=0x82,
S7AreaMK=0x83, S7AreaDB=0x84) but we were comparing against server area codes
(srvAreaPE=0, srvAreaPA=1, srvAreaMK=2, srvAreaDB=5).

This caused find_area_runtime() to always return NULL, resulting in the
callback returning 0 without filling any data - hence all reads returned zeros.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Added detailed logging to trace:
- Callback invocation with area code, operation, size
- Area/DB lookup results
- Buffer read operations with pointer values
- Read result data

This will help diagnose why reads are returning zeros.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The rm -rf /var/lib/apt/lists/* was a Docker-style optimization
that's counterproductive on regular systems - it forces users to
redownload the entire apt cache on every subsequent apt operation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Docker-specific optimizations like apt cache cleanup belong in the
Dockerfile, not the install script. This keeps install.sh generic
for all platforms while still optimizing Docker image size.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
S7Comm plugin with journal-based writes is now working correctly.
Removed temporary debug logging that was added for troubleshooting.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add journal buffer system for race-condition-free plugin writes
@thiagoralves
thiagoralves merged commit 72ef47b into main Jan 14, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant